♻️ Remove async import #12042

manuel-sommer · 2025-03-18T08:18:54Z

No description provided.

dryrunsecurity · 2025-03-18T08:19:30Z

DryRun Security Summary

The pull request removes deprecated asynchronous finding import features across multiple files, reducing security risks and simplifying system configuration by eliminating experimental code paths and potential concurrency-related vulnerabilities.

Expand for full summary

PR Summary: Removal of deprecated asynchronous finding import feature across multiple files, including documentation updates, code cleanup in importers, and configuration settings modifications.

Security Findings:

Removal of experimental async processing methods reduces potential race conditions and concurrency-related security risks
Eliminates configuration settings for async finding imports, simplifying the system's configuration
Reduces attack surface by removing deprecated and experimental code paths
Removes potential entry points for unintended behavior in import processes
Cleans up unused imports that could potentially introduce security complexities

No direct security vulnerabilities were introduced by these changes.

Code Analysis

We ran 7 analyzers against 6 files and 1 analyzer had findings. 6 analyzers had no findings.

Analyzer	Findings
Configured Codepaths Analyzer	7 findings

Overall Riskiness

🔴 Risk threshold exceeded.

We've notified @mtesauro, @grendel513.

View PR in the DryRun Dashboard.

valentijnscholten · 2025-03-19T20:50:20Z

Can I suggest to change the title to "Remove async import"?

manuel-sommer · 2025-03-19T20:59:38Z

Done

Maffooch · 2025-03-24T15:40:52Z

@manuel-sommer thank you for doing this! It will definitely save us some time in the future. We are planning to remove this functionality in the June release to provide folks enough awareness and time. The earliest we could merge this would be shortly after the May release

manuel-sommer · 2025-03-24T15:43:42Z

Sure, feel free to merge it later. :-)

github-actions · 2025-03-27T19:36:10Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

manuel-sommer · 2025-03-27T19:55:32Z

I will resolve the conflicts once this will be picked up again.

github-actions · 2025-04-15T07:32:32Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

dryrunsecurity · 2025-04-15T07:33:12Z

🔴 Risk threshold exceeded.

This pull request involves sensitive edits to multiple importer files in the dojo/importers directory, with potential implications for async import functionality, performance, and system configuration flexibility, including the removal of deprecated features and configurations.

⚠️ Configured Codepaths Edit in dojo/importers/endpoint_manager.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/endpoint_manager.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/base_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_reimporter.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/base_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_reimporter.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_reimporter.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/endpoint_manager.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/endpoint_manager.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/default_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

⚠️ Configured Codepaths Edit in dojo/importers/base_importer.py

Vulnerability	Configured Codepaths Edit
Description	Sensitive edits detected for this file. Sensitive file paths and allowed authors can be configured in `.dryrunsecurity.yaml`.

💭 Unconfirmed Findings (4)

Vulnerability	Potential Feature Deprecation Impact
Description	Removal of async import functionality in documentation file, which could cause compatibility issues for users who have not migrated away from async import, potentially leading to unexpected behavior or data import failures.

Vulnerability	Potential Performance Regression
Description	Synchronous replacement of async endpoint processing in endpoint_manager.py, which may impact performance for large endpoint volumes by removing distributed processing across Celery workers.

Vulnerability	Reduced Flexibility in Endpoint Processing
Description	Loss of dynamic configuration for endpoint processing due to removal of configurable async import settings, reducing system adaptability.

Vulnerability	Deprecated Feature Configuration Removal
Description	Elimination of deprecated configurations in settings.dist.py to prevent unintended behavior and potential misconfigurations with outdated experimental features.

We've notified @mtesauro.

All finding details can be found in the DryRun Security Dashboard.

github-actions · 2025-05-05T16:42:55Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-05-06T06:22:14Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

manuel-sommer · 2025-05-06T06:24:18Z

Hi @mtesauro and @Maffooch ,
I rebased. Ready to be reviewed.

mtesauro

Approved

docs/content/en/open_source/installation/running-in-production.md

valentijnscholten · 2025-05-08T13:39:51Z

dojo/settings/settings.dist.py

@@ -273,12 +273,6 @@
    DD_RATE_LIMITER_ACCOUNT_LOCKOUT=(bool, False),
    # when enabled SonarQube API parser will download the security hotspots
    DD_SONARQUBE_API_PARSER_HOTSPOTS=(bool, True),
-    # when enabled, finding importing will occur asynchronously, default False
-    # This experimental feature has been deprecated as of DefectDojo 2.44.0 (March release). Please exercise caution if using this feature with an older version of DefectDojo, as results may be inconsistent.
-    DD_ASYNC_FINDING_IMPORT=(bool, False),


If True, would it make sense to create an announcement banner or other alert to notify the users / admins that they are using a feature that is no longer present?

I don't have the time right now to implement this.

valentijnscholten

Nice cleanup, left two comments which are not really blocking the merge, just some thoughts.

♻️ Deprecate async import

ce489cc

github-actions bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR docs labels Mar 18, 2025

manuel-sommer added 2 commits March 18, 2025 10:28

simplify

e29727b

update

a3c0126

manuel-sommer closed this Mar 18, 2025

manuel-sommer reopened this Mar 18, 2025

fix unittest

1cc9fe6

manuel-sommer changed the title ~~♻️ Deprecate async import~~ ♻️ Remove async import Mar 19, 2025

github-actions bot added the conflicts-detected label Mar 27, 2025

Maffooch added this to the 2.47.0 milestone Apr 7, 2025

Merge branch 'dev' into remove_async_import

4a6e042

manuel-sommer requested review from Maffooch and mtesauro as code owners April 15, 2025 07:32

github-actions bot removed the conflicts-detected label Apr 15, 2025

github-actions bot added the conflicts-detected label May 5, 2025

Merge branch 'dev' into remove_async_import

8ca7f36

github-actions bot removed the conflicts-detected label May 6, 2025

mtesauro approved these changes May 8, 2025

View reviewed changes

add docs

9b97d8b

valentijnscholten reviewed May 8, 2025

View reviewed changes

docs/content/en/open_source/installation/running-in-production.md Show resolved Hide resolved

valentijnscholten reviewed May 8, 2025

View reviewed changes

Maffooch requested a review from hblankenship May 8, 2025 15:17

Maffooch approved these changes May 9, 2025

View reviewed changes

update

307643a

manuel-sommer requested a review from valentijnscholten May 15, 2025 10:07

Maffooch requested a review from dogboat May 22, 2025 04:27

valentijnscholten approved these changes May 26, 2025

View reviewed changes

♻️ Remove async import #12042

Are you sure you want to change the base?

♻️ Remove async import #12042

Uh oh!

Conversation

manuel-sommer commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dryrunsecurity bot commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DryRun Security Summary

Code Analysis

Overall Riskiness

Uh oh!

valentijnscholten commented Mar 19, 2025

Uh oh!

manuel-sommer commented Mar 19, 2025

Uh oh!

Maffooch commented Mar 24, 2025

Uh oh!

manuel-sommer commented Mar 24, 2025

Uh oh!

github-actions bot commented Mar 27, 2025

Uh oh!

manuel-sommer commented Mar 27, 2025

Uh oh!

github-actions bot commented Apr 15, 2025

Uh oh!

dryrunsecurity bot commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 Risk threshold exceeded.

Uh oh!

github-actions bot commented May 5, 2025

Uh oh!

github-actions bot commented May 6, 2025

Uh oh!

manuel-sommer commented May 6, 2025

Uh oh!

mtesauro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

valentijnscholten May 8, 2025

Choose a reason for hiding this comment

Uh oh!

manuel-sommer May 15, 2025

Choose a reason for hiding this comment

Uh oh!

valentijnscholten left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

manuel-sommer commented Mar 18, 2025 •

edited

Loading

dryrunsecurity bot commented Mar 18, 2025 •

edited

Loading

dryrunsecurity bot commented Apr 15, 2025 •

edited

Loading